The E-business marketplace is highly competitive. A strong online marketing strategy is one key to a successful e-commerce business. Site operators often need to know what kind and combinations of content, design and flow will most appeal to their customers and differentiate their offers from those of their competitors. More importantly, they want to know which combinations produce the best browse-to-buy conversion rate. Generally, system operators want to test for one or more of the following reasons:                Determine Best Design        Find Optimized Pricing        Test New Ideas Before Making Full Commitments        Increase Conversion Rates        Offer Best Products & Services        Match Promotions with Customer Wants, Needs        Improve Retention Rates        Maximize Advertising Effectiveness        Drive Top-Line Revenue Growth        Localize for the Right Audience        
A need exists for a rich set of site testing remote control tools that site operators and marketers can use to test and optimize site performance across a number of areas: pricing, cross sell/up sell products, content displays, page flow, product offers and more.
One of the most common types of web site testing is known as serial testing (shown in FIG. 1) where a webpage design is tested one week in one configuration and then switched to another design the following week and tested.
Another type of test is A/B testing as shown in FIG. 2. A/B testing has several advantages over serial testing; most particularly, the availability of a controlled environment leading to valid data that can be used to make optimal business decisions. A/B testing provides an effective way to optimize online store performance because it uses principles of experimental design, is metrics-driven, objective, and more efficient than other site optimization techniques.
In its simplest form, A/B testing allows the user to randomly display several content, pricing or promotional offers simultaneously on the storefront while testing the impact of each approach. For example, if a user wanted to determine the best use of a promotional banner area on the store home page, s/he might set up two different creative designs and show each to 50% of the store traffic over a period of time. A/B testing can be configured with two or more test cells, and split into any kind of traffic segmentation the user deems appropriate. For instance, if a user knew that a particular approach would be more popular than the others, s/he could set up the more popular version to run for a larger percentage of traffic in order to maximize revenue while completing the test. The user would then measure the close ratio (conversion rate) and sales revenue impact of each design.
Another benefit of this approach is that all other factors are controlled (referring traffic source, other store promotions, etc.) because traffic is randomly assigned to each of the test designs. This strategy yields an objective measurement; data with which real quality decisions may be made. Additional benefits are faster test results and statistically accurate tracking which in turn leads to higher quality business decisions.
Data from each A/B test cell (a branch or version of the test) becomes the basis for which test cell is determined the strongest or highest performing. Some common measures of test performance for an online store are: close ratio, sales revenue, number of sales, average order size, and sales per unique visitor. Close ratio and sales per unique visitor are usually the most important metrics for A/B testing because they show the actual revenue impact of a test and allow the user to compare the results of each test cell even if one test cell received more traffic or views than another.
When reading test results, it is critical to evaluate the statistical significance of the data in the test. The statistical significance is a measure of whether the lift or gain seen in a particular test cell is actually statistically valid within a certain confidence level. Think of this as similar to election night results that are within a certain margin of error. If the results are too close to call and fall within the margin of error, the user may not be able to make a determination of the winner. Statistical significance is similar, in that if test cells are too close across close ratio and revenue per visitor terms, it is difficult to make a good decision about which test cell is the winner because the difference in performance of any particular cell is not statistically different than the other cells.
A/B testing often requires creation of static web pages that further require development resources in order to create links between the test offer and the shopping cart. These links ensure that the shopper views the offer and receives the attributes (i.e. pricing) contained in it rather than those of the standard offer.
A need exists for improved testing tools for e-commerce system users. The present invention provides a solution to these needs and other problems, and offers other advantages over the prior art.